The increasingly limiting bottleneck to information flow in ultra-mobile computing results from, among other things, the lack of efficient user interfaces on these devices. In comparison to desktop or laptop computers, today's handsets and tablet PCs are far more cumbersome to use. One solution is “shape writing” where a user inputs shorthand gestures that are defined by a virtual keyboard layout. Such shorthand gestures are continuous strokes that are recognized by a continuous stroke recognizer based on a virtual keyboard. Examples of such systems are described in Kristensson, P. O., Design and Evaluation of a Shorthand-Aided Soft Keyboard, Master's thesis, Linköping University (2002); U.S. Pat. No. 7,251,367, System and method for recognizing word patterns based on a virtual keyboard layout (Zhai); U.S. Pat. No. 7,098,896, System and method for continuous stroke word-based text input (Kushler et al.); and Kristensson, P. O. and Zhai, S., SHARK.sup.2: A Large Vocabulary Shorthand Writing System for Pen-based Computers, Proc. 17.sup.th Annual ACM Symposium on User Interface Software and Technology 2004, 43-52 (2004). Such systems and methods do not provide previews based upon uncompleted strokes; do not provide preview or correction displays integrated in a virtual keyboard (for example, in Kushler et al., a display is incorporated in a word processing application, and in Kristensson, P. O. and Zhai, S., a user must utilize a pull-down menu); and do not provide for deletion of words in such a preview or correction display.
Similarly, U.S. Pat. No. 7,706,616, System and method for recognizing word patterns in a very large vocabulary based on a virtual keyboard layout (Kristensson et al.) and U.S. Pat. No. 7,382,358, System and Method for Continuous Stroke Word-Based Text Input (Kushler et al.), present systems or methods for recognizing continuous strokes based on a virtual keyboard layout, but do not provide a preview of the recognized command for an uncompleted stroke; an integrated correction display; or an option to delete or cancel an output word by selecting a representation of the current word in a selection display. United States Patent Application 20050114115, Typing accuracy relaxation system and method in stylus and other keyboards (Karadis et al.), discloses a system and method to analyze tapping patterns and automatically correct mis-tapped words, but does not use a continuous stroke recognizer. Arvo, J. and Novins, K., Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes. Proceedings of the 17.sup.th Annul. ACM Symposium on User Interface Software and Technology 2000: 73-80 (2000), presents a software system to recognize pen-gestures while a user is articulating the pen-gesture, but works with a limited set of pen-gestures. See, also, Zhai, S. and Kristensson, P. O., Shorthand Writing on Stylus Keyboard. Proceedings of the ACM Conference on Human Factors in Computing Systems 2003, 17-24 (2003).
Briefly, shorthand gesturing on a virtual keyboard works by having a user gesturing a pen or finger trace on a virtual keyboard layout (such as the virtual keyboard layout illustrated in FIG. 3 of U.S. Pat. No. 7,251,367 (Zhai), which patent is incorporated herein by reference). When the user lifts up the pen or finger, the system recognizes the pen or finger trace. The pen or finger trace is then compared with the geometric trace or pattern connecting the letter keys of a template word in a lexicon. This comparison is repeated for all words in a lexicon. The word that most closely matches the user's pen or finger trace is returned as the recognized word.
Some users may not feel confident when shorthand gesturing because the input interface is new to them. Such users, including novice users, simultaneously need to learn a new shorthand gesture text input interface and verify that they are on the right track.
Another problem with existing shorthand gesturing is that no recognition or feedback from the system is communicated to the user while the user is producing a shorthand gesture to indicate the user is or is not on the right track.
Existing shorthand gesturing does not provide for repeated and/or contemporaneous recognition of a user's pen or finger trace. In addition, there is a need to avoid situations in which a recognized word is sent to the associated application (such as a word processing application or e-mail client) without confirmation that it is the word intended by the user, and a need to provide a user with the opportunity to cancel the process if the user has made a mistake.
Users may also want to send computer system commands such as “Copy,” “Print” and “Track Changes” with a shorthand gesture. In this regard, reference is made to U.S. Pat. No. 7,487,461, System and method for issuing commands based on pen motions on a graphical keyboard (Zhai et al.), which does not provide a preview of an uncompleted stroke, or a display integrated with a virtual keyboard, or an option to delete an unintended word.
When a user is gesturing a shorthand gesture, the need for a preview is also desired. In addition, computer system commands perform operations that affect the user's application and can be destructive to the user's work. For example, the computer system command “Close” may close the user's application and result in data loss for the user.
Clearly there is a need for both novice and expert users of shorthand gesturing for a preview of the word or system command input by a user as a pen or finger trace, and for other solutions as set forth herein.